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Abstract 

Evolutionary graph theory is a well established framework for modelling the evolution of social behaviours in structured 
populations. An emerging consensus in this field is that graphs that exhibit heterogeneity in the number of connections 
between individuals are more conducive to the spread of cooperative behaviours. In this article we show that such a 
conclusion largely depends on the individual-level interactions that take place. In particular, averaging payoffs garnered 
through game interactions rather than accumulating the payoffs can altogether remove the cooperative advantage of 
heterogeneous graphs while such a difference does not affect the outcome on homogeneous structures. In addition, the 
rate at which game interactions occur can alter the evolutionary outcome. Less interactions allow heterogeneous graphs to 
support more cooperation than homogeneous graphs, while higher rates of interactions make homogeneous and 
heterogeneous graphs virtually indistinguishable in their ability to support cooperation. Most importantly, we show that 
common measures of evolutionary advantage used in homogeneous populations, such as a comparison of the fixation 
probability of a rare mutant to that of the resident type, are no longer valid in heterogeneous populations. Heterogeneity 
causes a bias in where mutations occur in the population which affects the mutant's fixation probability. We derive the 
appropriate measures for heterogeneous populations that account for this bias. 
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Introduction 

Population structure has long been known to affect the outcome 
of an evolutionary process [1—4]. Evolutionary graph theory has 
emerged as a convenient framework for modelling structured 
populations [4,5]. Individuals reside on vertices of the graph and 
the edges define the interaction neighbourhoods. 

A variety of processes have been investigated on a number of 
graph classes. However, few analytical results exist in general, 
since an arbitrary graph may not exhibit sufficient symmetry to aid 
calculations. The most general class of graphs for which analytical 
results are known is the class of homogeneous (vertex-transitive) 
graphs. Such a graph G has the property that for any two vertices 
Vj and Vj there exists a structure-preserving transformation g of G 
such that g(Vj) = Vj. It is worth noting that not all regular graphs 
are homogeneous; an extreme example is the Frucht graph [6], 
which is regular of degree 3 and has only the trivial symmetry. 
Intuitively, this class consists of graphs that "look" the same from 
any vertex. The amount of symmetry in such graphs has allowed 
for a complete set of analytical results for restricted types of 
evolutionary processes and weak selection [7-9]. Despite the 
tractability of calculations on homogeneous graphs, natural 
population structures are seldom homogeneous. Therefore it is 
important to understand the effects of heterogeneous population 
structures on evolutionary processes [4,8,10] and, in particular, on 
the evolution of cooperation. 



In the simplest case there are two strategic types: cooper- 
ators that provide a benefit b to their interaction partner at 
some cost c to themselves (b>c>0), whereas defectors provide 
neither benefits nor incur costs. This basic setup is known as an 
instance of the prisoner's dilemma and reflects a conflict of 
interest because mutual cooperation yields payoff b — c>0 and 
hence both parties prefer this outcome over mutual defection, 
which yields a payoff of zero. However, at the same time 
each party is tempted to defect in order to avoid the costs 
of cooperation. The temptation of increased benefits for 
unilateral defection thwarts cooperation - to the detriment of 
all. This conflict of interest characterizes social dilemmas 
[11,12]. 

More general kinds of interactions between two individuals 
and two strategic types, A and B, can be represented in the 
form of a 2x2 payoff matrix as in Table 1 . The payoffs 
garnered from these game interactions affect an individual's 
expected number of offspring by altering their propensity to 
have offspring (their fitness) or their survival. The expected 
number of offspring is determined by the fitness of the 
individuals and some population updating process, which will 
be made precise in the next section. The offspring produced 
during the population update have the potential to change the 
strategy composition of the population. An increase in the 
abundance of one strategy over a sufficiently large time scale 
indicates that strategy is favoured by evolution. 
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Author Summary 

Understanding the evolution of cooperation is a persistent 
challenge to evolutionary theorists. A contemporary take 
on this subject is to model populations with interactions 
structured as close as possible to actual social networks. 
These networks are heterogeneous in the number and 
type of contact each member has. Our paper demon- 
strates that the fate of cooperation in such heterogeneous 
populations critically depends on the rate at which 
interactions occur and how interactions translate into the 
fitnesses of the strategies. We also develop theory that 
allows for an evolutionary analysis in heterogeneous 
populations. This includes deriving appropriate criteria 
for evolutionary advantage. 

It can be shown, for replicator dynamics, for example [13,14], 
that any payofT matrix can be reduced to the matrix in Table 1 
without loss of generality because adding a constant term to the 
payoff matrix does not affect the dynamics and multiplying the 
payoffs by a positive factor merely rescales the time. Therefore we 
can always shift the payoffs such that _S-i?-encounters return a 
payoff of zero and scale all other payoffs such that ^4-^4-encounters 
yield a payoff of 1 . In the Averaged versus Accumulated Payoffs 
section, we show that the generality of the matrix in Table 1 
extends to other forms of stochastic dynamics in finite populations 
based on the frequency dependent Moran process [15]. 

The (additive) prisoner's dilemma introduced before corre- 
sponds to the special case with S= —c/(b — c) and T = bj(b — c). 
Rescaling the payoff matrix in Table 1 by b — c yields the 
traditional form, Table 2. More generally, the prisoner's dilemma 
requires S < 0 and T > 1 to result in the characteristic conflict of 
interest outline above. The special case of the additive prisoner's 
dilemma, Table 2, effectively reduces the game to a single 
parameter with T= 1 — S (and S<0). Moreover it has the special 
property that when an individual changes its strategy, the payoff 
gain (or loss) is the same, regardless of the opponents' strategy - 
the so-called equal-gains-from-switching property [16]. 

In the absence of structure, cooperators dwindle and disappear 
in the prisoner's dilemma. In contrast, structured populations 
enable cooperators to form clusters, which ensures that cooper- 
ators more frequently interact with other cooperators than they 
would with random interactions [17,18]. Such assortment between 
cooperators is essential for the survival of cooperation [19]. 

In heterogeneous graphs not all vertices have the same number 
of connections and hence the fitnesses of individuals may be based 
on different numbers of interactions. Because of this, some vertices 
are more advantageous to occupy than others. However, which 
sites are favourable depends on the type of population dynamics. 
In particular, for the Moran process in structured populations it is 
important to distinguish between birth-death and death-birth 
updating [10,20,21], i.e. whether first an individual is randomly 



Table 1. The payoff matrix for a general 2 by 2 strategy 
game. 
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A 1 
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B T 
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Here 5 and Tare real numbers. 
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selected for reproduction with a probability proportional to its 
fitness and then the clonal offspring replaces a (uniformly) 
randomly selected neighbour - or, if first an individual is selected 
at random to die and then the vacant site is repopulated with the 
offspring of a neighbouring individual with a probability 
proportional to its fitness. Even in homogenous populations the 
sequence of events is of crucial importance but becomes even more 
pronounced in heterogenous structures [10,20]. 

In order to illustrate that the population dynamics may bestow 
an advantage to individuals occupying certain sites in a 
heterogeneous population, consider neutral evolution, where game 
payoffs do not affect the evolutionary process and all individuals 
have the same fitness. For birth-death updating every individual is 
chosen to reproduce with the same probability but neighbours of 
individuals with few connections are replaced more frequently. 
Hence vertices with fewer neighbours are more favourable than 
those with many connections. Conversely, for death-birth updat- 
ing every individual has the same expected life time but highly- 
connected individuals, or, hubs, get more frequently a chance to 
produce offspring, since one of their many neighbours dies, and 
are thus more favourable than vertices with few neighbours [21- 
23]. A simple example of this is a 3-line graph, one central vertex 
connected to two end vertices. In the birth-death process, the 
central vertex is replaced with probability 2/3, while either end 
vertex is replaced with probability 1/6, while in the death-birth 
process, the central vertex replaces either end vertex with 
probability 2/3 and either end replaces the centre with probability 
1/6 [21]. The upshot is, even though the fitness of all individuals is 
the same, the effective number of offspring produced depends on 
the dynamics as well as an individual's location in the population. 

The intrinsic advantage of some vertices over others can be 
further enhanced through game interactions leading to differences 
in fitness that depend on an individual's strategy as well as its 
position on the graph. For example, a cooperator occupying a 
favourable vertex can more easily establish a cluster of coopera- 
tors, which creates a positive feedback through mutual increases in 
fitness. Conversely, a favourable vertex also supports the formation 
of a cluster of defectors but this results in a negative feedback and 
lowers the fitness of the defector in the favourable vertex. The fact 
that heterogeneity can promote cooperation was first observed for 
the prisoner's dilemma and snowdrift games [24,25] and has 
subsequently been confirmed for public goods games [26,27]. 
However, the detailed effects not only crucially depend on the 
dynamics but also on how fitnesses are determined. For example, 
heterogenous population structures favour cooperation if payoffs 
from game interactions are accumulated but that advantage 
disappears if payoffs are averaged [28-30]. 

The effects of population structure on the outcome of 
evolutionary games is sensitive to a number of factors: population 
dynamics [10,20,31], translation of payoffs into fitness [28,30,32- 
35], the diversity of players [27,34,36], and the type of game 
played - for example, spatial structure tends to support 
cooperation in the prisoner's dilemma but conversely, in the 
snowdrift game, spatial structure may be detrimental [37]. 
Macroscopic features of the evolutionary process on the level of 
the population, such as frequency and distribution of cooperators, 
are determined by microscopic processes on the level of 
individuals. In the current article, we discuss some of these 
microscopic processes, such as averaging and accumulating 
payoffs, and the rate at which interactions take place, and 
illustrate how they affect an evolutionary outcome. Crucially, we 
also illustrate that the conditions for evolutionary advantage 
commonly found in the literature are not applicable to evolution in 
finite, heterogeneous populations. We modify these conditions and 
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Table 2. The payoff matrix for an additive prisoner's dilemma 
game. 



that is, <5«1, then 
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develop a general framework to determine evolutionary advantage 
in finite, heterogeneous populations. 

The manuscript is organized as follows. Sections "Accumulated 
and Averaged Payoffs" and "Criteria for Evolutionary Success" 
create a critical synthesis of the existing literature concerning 
evolution on heterogeneous graphs. In these sections we extend 
existing results to general 2x2 games and focus on an imitation 
process. We also discuss the inapplicability of approaches used in 
homogeneous populations and present our novel conditions for 
evolutionary success in heterogeneous populations. Interspersed in 
these sections are new observations and results that aid in 
establishing a consistent framework on which we base further 
novel results presented in the section "Stochastic Interactions and 
Updates". 

Results 

Accumulated Versus Averaged Payoffs 

In heterogenous population structures individuals naturally 
engage in different numbers of interactions. This renders 
comparisons of the performances of individuals more challenging. 
One natural approach is to simply accumulate the game payoffs. 
This clearly puts hubs with many neighbours in a strong position 
as scoring many times even a small payoff may still exceed few 
large payoffs. To avoid this bias in favour of hubs, game payoffs 
can be averaged. Interestingly, these two approaches not only play 
a decisive role for the evolutionary outcome but also entail 
important biological implications. In this section we extend 
previous work on payoff accounting [29] to general 2x2 games 
and provide a thorough discussion of why different payoff 
accounting schemes can result in markedly different evolutionary 
outcomes. 

Consider two different ways to translate the total, accumulated 
payoffs 71/ of an individual i into its fitness/: 



accumulated: fi = e 



,8m 



averaged : fi = e 



(la) 



(lb) 



where 3 > 0 denotes the strength of selection and n, is the number 
of interactions experienced by i. The limit <5-»0 recovers the 
neutral process, where selection does not act. Note that the payoff 
matrix in Table 1 can still be used without loss of generality 
because adding a constant K merely changes the (arbitrary) 

baseline fitness from 1 to e^ K and multiplying the payoffs by X is 
identical to simply changing the selection strength to 3 k. 

The exponential form of fitness in the above equations is 
mathematically convenient since it guarantees that the fitness is 
always positive, irrespective of the strength of selection and payoff 
values. It is worth noting that if the strength of selection is weak, 



accumulated : fj = e^ 71 ' x 1 + 3m + 0(3), 



3^ 

averaged :fi = e n > « 1 • 



-5- + 0(3), 

Hi 



(2a) 



(2b) 



which represents another common form for fitness found in the 
literature [8], 

Homogenous populations. In the past, details of the 
payoff accounting have received limited attention, or the two 
approaches have been used interchangeably, because they 
yield essentially the same results for traditional models of 
spatial games, which focus on lattice populations [4,38] or, 
more generally, on homogenous populations [8,10,39]. In fact, 
the difference in payoff accounting reduces to a change in the 
selection strength because in homogenous populations each 
individual has the same degree d, = d (number of neighbours) 
and hence, on average, the same number of interactions n per 
unit time. If each individual interacts with all its neighbours 
then n = d. Thus, the only difference is that the selection 
strength for accumulated payoffs is M-times as strong as for 
averaged payoffs. 

Therefore, in homogenous populations all individuals engage in 
the same number of interactions per unit time and consequendy 
accumulating or averaging payoffs merely affects the strength of 
selection. Naturally, the converse question arises — are uniform 
interaction rates restricted to homogenous graphs? Or, more 
generally, which class of graphs supports uniform interaction 
rates? 

To answer this question, let us consider an arbitrary graph G 
with adjacency matrix W = [vfa] where Wh > 0 indicates the weight 
or the strength of the (directed) edge from vertex i to j. Wh > 0 if 
vertex i is connected to j and w« = 0 if it is not. For example, the 
natural choice for the edge weights on undirected graphs is 
Wjj = l/dj. That is, all di edges leaving vertex i have the same 

weight and hence Wy = 1 for all i. 

An individual on vertex i is selected to interact with vertex j with 
a probability proportional to Wy. In this case we say vertex / has 
initiated the interaction. Interactions with self are excluded by 
requiring Wa = 0. If there are M interactions per unit time, then 
the average number of interactions «/ that vertex / engages in is 
given by 



■ M 



E,=i^+E,v 



(3) 



where the fraction indicates the probability that vertex i 
participates in one particular interaction either by initiating it 
(first sum in numerator) or initiated by neighbours of i (second sum 
in numerator). On average each individual engages in 2M/N 
interactions. Note that the factor 2 enters because each interaction 
affects two individuals. Therefore, a graph structure results in 
uniform interaction rates if and only if 



ELk- 



2 
N 



(4) 
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holds for every vertex ;', or equivalently, if y ( (wy + Wjj) = Co 
for all i where Co is an arbitrary positive constant. 



If the sum of the weights of all 



edges leaving vertex i, 

N 



y IV/,- = C\ >0, is the same for all i then ^ <;y- \ w kj = N-C\ 

and Eq. (4) requires that the sum of the weights of all incoming 

edges, Wjj = C\, for all i, as well to ensure uniform 

interaction rates. The class of graphs that satisfies the condition 

. _ j Wy = . _ j Wji = C\ for all i are called circulations [5] and, 

in the special case with C\ = 1, the adjacency matrix is doubly 
stochastic such that each row and column sums to 1. A more 
generic representative of the broad class of circulation graphs is 
shown in Figure 1 but this does not include heterogenous graphs 
such as scale-free networks. 

In order to illustrate that the number of interactions experi- 
enced by an individual depends on which vertex they reside, let us 
consider a mean-field approximation for sufficiently large 
networks, based on the degree distribution p(d) and degree- 
degree correlations p(dj\di) [40]. Specifically, p(d) denotes the 
probability of a randomly chosen vertex having degree d, and 
p(dj\di) denotes the conditional probability of a vertex / with 
degree dj connected to vertices j with degree dj. With this 
notation, the connectivity between a vertex of degree dj and 
another vertex of degree dj is djp(d,\dj). Further averaging this 
quantity over all vertices having degree dj, we obtain Wy, which 
indicates the weight of the connection between a vertex of degree 
di and another vertex of degree dj, 



djp(dj\dj) 
Np{di) ■ 



(5) 



The formula above omits higher-order correlations than two-point 
correlations, and works for large, sparse networks (N » 1 and 
N»di for all i) [41]. In this case, Wy can be interpreted as the 
probability that vertices i and j are connected at the mean-field 
level. 

For random, undirected, and degree-uncorrelated graphs, 
p(di\dj) does not depend on dj and is thus given by djp(di)jd, 
where d is the average degree of the network, d= J2i d '/ N - This 
applies even if the network is not sparse. Accordingly, Wy can be 
simplified: 



djdi 



Inserting into Eq. (4) yields 



rti = M 



2_d, 
Nd' 



(6) 



(7) 



Hence, the number of interactions of one vertex scales linearly 
with its degree. 

Similarly, each vertex can initiate the same number of 
interactions, m. Then, with probability Wjt/dj the neighbouring 
vertex j initiates an interaction with /: 




Figure 1. A representative example of the broad class of circulation graphs. Note that the weights of edges entering as well as those 

leaving any vertex all sum to 1. 

doi:10.1371/journal.pcbi.1003567.g001 
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= m 1 + 



N-ldi 



( 



\ 



7=1 



1 , 

2 2fj+fi' 



(9) 



where /y and ft are the fitnesses of i and _/. This variant is 
(8) convenient as it includes an appropriate normalization factor 
and hence works regardless of how the fitnesses are calculated. 
In particular, for exponential payoff-to-fitness mapping (see 
Eq. (1)) the imitation rule, Eq. (9), recovers the Fermi-update 
[45]: 



Again, vertices with a degree greater (less) than the average degree 
are expected to have more (fewer) interactions than on average. 
Interaction rates on various heterogenous networks are shown in 
Figure 2. As shown in Figures 2a-c, this approximation above 
works well for a variety of networks where degree of adjacent 
vertices are uncorrelated. However, when the network is strongly 
degree-correlated, like the two-star graphs [27,34], this approxi- 
mation works poorly (also see Figure 2d for such an example of 
highly clustered scale free networks). In this case, we may use 
Eq.(5) to calculate Wy as long as the function for degree-degree 
correlations, p(di\dj), is explicitly known. 

This indicates that on undirected graphs uniform interaction 
rates can be achieved only on regular graphs, where all vertices 
have the same number of neighbours. 

Heterogenous populations. In recent years the focus has 
shifted from homogenous populations to heterogenous structures 
and, in particular, to small-world or scale-free networks because 
they capture intriguing features of social networks [42] . On these 
structures the accounting of payoffs becomes important and, 
indeed, a crucial determinant of the evolutionary outcome. If 
payoffs are accumulated, heterogenous structures further promote 
the evolution of cooperation [24,25,27,36]. In contrast, averaging 
the game payoffs can remove the ability for scale-free graphs to 
sustain higher levels of cooperation [28-30]. 

So far our discussion has focused on interactions between 
individuals and the translation of payoffs into fitness. The next step 
is to specify how differences in fitness affect the population 
dynamics. The most common updating rules in evolutionary 
games on graphs fall into three categories: Moran birth-death and 
death-birth, and imitation processes. The evolutionary outcome 
can be highly sensitive to the choice of update rule. For example, 
supposing weak selection, cooperation in the prisoner's dilemma 
may only thrive under death-birth but not under birth-death 
updating [8,10,20]. 

In heterogenous populations the range of payoffs depends on 
the payoff accounting: if payoffs are averaged, the range is 
determined by the maximum and minimum values in the payoff 
matrix but if payoffs are accumulated the range additionally 
depends on the size and structure of the population. In particular, 
this difference may also affect the updating rule: for example, the 
pairwise comparison process l/2 + (//— /})/a represents the 
probability that vertex i adopts the strategy of vertex j based on 
their fitnesses of fi, fj, respectively [43,44]. This represents an 
imitation process where a denotes a sufficiently large normaliza- 
tion constant to ensure that the expression indeed remains a 
probability. Since a needs to be at least twice the range of possible 
fitness values, a generic choice of a becomes impossible for 
accumulated payoffs. 

Here we focus on a related imitation process where an 
individual i is chosen at random to reassess its strategy by 
comparing its performance to a randomly chosen neighbour j. 
Individual i then imitates the strategy of j with probability 



accumulated : =7 r , ( 1 



averaged: -. r-, (10b) 

l+e \ n J 

For a comparison between averaged and accumulated 
payoffs in homogenous and heterogenous populations, see 
Figure 3. 

On a microscopic level averaging or accumulating payoffs in 
heterogenous populations turns out to have important biolog- 
ical implications: when averaging payoffs, individuals play 
different games depending on their location on the graph, 
whereas for accumulated payoffs everyone plays the same 
game but at different rates - again based on the individuals' 
locations. These intriguing differences are illustrated and 
discussed for the simplest heterogenous structure, the star 
graph, in a subsequent section. First we develop a framework 
that aids in analyzing an evolutionary process in heteroge- 
neous, graph-structured populations. 

Criteria for Evolutionary Success 

In order to determine the evolutionary success of a strategic type 
in a finite population we consider three fixation probabilities: 
Pa>Pb an d Po- The first, p A , indicates the probability that a single 
A type in an otherwise B population goes on to supplant all 5s, 
while the second, p B , refers to the probability of the converse 
process where a single B type takes over a population of A types. 
These fixation probabilities are important whenever mutations can 
arise in the population during reproduction or through errors in 
imitating the strategies of others. The last probability, p 0 , denotes 
the fixation probability of the neutral process, which is defined as 
the dynamic in a population with vanishing selection, <5 = 0. In 
such a case the game payoffs do not matter and everyone has the 
same fitness. Based on these fixation probabilities two distinct and 
complementary criteria are traditionally used to measure evolu- 
tionary success [15,20]: 

(i) Type A is said to have an evolutionary advantage or is favoured 
if 

Pa>Pb (11) 

holds. If mutations, or errors in imitation, are rare the mutant 
has disappeared or taken over the entire population before 
the next mutation occurs. We can then view the population 
dynamic as an embedded Markov chain transitioning 
between two states: all-^4 and al\-B. Denote the proportion 
of time spent in the state all-^4 (respectively, ail-B) by Ta 
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Number of neighbors 

Figure 2. Average number of interactions as a function of the degree of the vertex for different types of random heterogenous 
population structures: (A) Erdos-Renyi random graphs [53], (B) Newman-Watts small-world networks [54. (C) Barabasi-Albert scale-free networks 
[42], and (D) Klemm-Eguiluz highly-clustered scale-free networks [55]. All graphs have size ^ = 1000 and an average degree of r5=10. At each time 
step a randomly chosen individual interacts with a randomly selected neighbour. The average number of interactions is shown for simulations (blue 
dots) and an analytical approximation for graphs where the degrees of adjacent vertices are uncorrelated (red line, see Eq. (8)). 
doi:10.1371/journal.pcbi.1003567.g002 

(resp. T B ). Together, T A and T B are known as the stationary (ii) Type A is a beneficial mutation if 

distribution of the Markov chain and satisfy the balance 

equation 

Pa>Po ( 14a ) 

TbHaPa = TaHbPb, (12) holds. Similarly, if 



where ji A (fi B ) is the probability an A (B) appears in the aft-B 
(all-yl) population. For homogeneous populations, or if 
mutations are not tied to reproduction or imitation events, 
Pa = Pb an d so Eq. (12) reads 

T B p A = T A p B . (13) 

Hence, if p A > p B then T A > T B , which captures the notion 
of A having an advantage over B. If the inequality, Eq. (1 1), 
is reversed then type B has the advantage. 



p B >p 0 (14b) 

holds, the B type is a beneficial mutation. Note that, in 
general, Eqs. (14a) and (14b) are not mutually exclusive. 
A and B types may simultaneously be advantageous 
mutants - in co-existence games, S>0,T>1, such as the 
snowdrift game - or both disadvantageous - in coordi- 
nation games, S <0,T <1, such as the stag-hunt game. 
However, for payoff matrices that satisfy equal-gains- 
from-switching, such as Table 2, p A >Po implies p B <Po 
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Figure 3. Average fraction of strategy . / for accumulated (top row) versus averaged (bottom row) payoffs in homogenous (left 
column) and heterogeneous (middle column) populations as well as the difference between them (right column) as a function of 
the game parameters S and T (see Table 1). In each panel the four quadrants indicate the four basic types of generalized social dilemmas: 
prisoner's dilemma (upper left), snowdrift or co-existence games (upper right), stag hunt or coordination games (lower left) and harmony games 
(lower right). Homogenous populations are represented by 50x50 lattices with von Neumann neighbourhood (degree d = 4) and heterogenous 
populations are represented by Barabasi-Albert scale-free networks (size N = 2500, average degree d = 4). The population is updated according to the 
imitation rule Eq. (9). The colours indicate the equilibrium fraction of strategy A (left and middle columns) ranging from A dominates (blue), equal 
proportions (green), to B dominates (red). Increases in equilibrium fractions due to heterogeneity are shown in blue shades (right column) and 
decreases in shades of red. The intensity of the colour indicates the strength of the effect. Accumulated payoffs in heterogenous populations shift the 
equilibrium in support of the more efficient strategy A except for harmony games where A dominates in any case (bottom right quadrant). 
Conversely, for averaged payoffs the support of strategy A is much weaker and even detrimental for T < 1 + S. Parameters: initial configuration is a 
random distribution of equal proportions of strategies A and B; each simulation run follows 1.610 7 updates and the equilibrium frequency of A is 
averaged over the last 2.510 6 updates; results are averaged over 500 independent runs; for scale-free networks the network is regenerated every 50 
runs. No mutations occured during the simulation run. 
doi:1 0.1 371 /journal.pcbi.1 003567.g003 



and vice versa in unstructured populations or for weak 
selection [46]. 

The above conditions (11) and (14) are based on the implicit 
assumption of homogenous populations or averaged payoffs and 
randomly placed mutants. In the present context of heterogenous 
populations and with mutants explicitly arising through errors in 
reproduction or imitation, both conditions require further scrutiny 
and appropriate adjustments. 

The first condition implicitly assumes that an A mutant appears 
in a monomorphic B population with the same probability as a B 
mutant in a monomorphic A population. However, in heterog- 
enous populations with accumulated payoffs this is not necessarily 
the case. Even in monomorphic states hubs may have a higher 
fitness and hence are more readily imitated, or reproduce more 



frequently, than low degree vertices. This can result in a bias of the 
rates fJ-^^s at which A and B mutants arise. Thus, the condition 
for evolutionary advantage, Eq. (1 1), must read 



PaPa>PbPb- 



(15) 



In general, ji A and fi B depend on the population structure as well 
as the payoffs and their accounting. The star structure serves as an 
illustrative example in the next section. 

Similarly, the second condition also needs to be made more 
explicit. In general, to determine whether a mutation is beneficial 
its fixation probability should exceed the probability that in the 
corresponding monomorphic population one particular individual 
eventually establishes as the common ancestor of the entire 



PLOS Computational Biology | www.ploscompbiol.org 



7 



April 2014 | Volume 10 | Issue 4 | e1 003567 



Evolutionary Dynamics in Heterogenous Populations 



population. We denote these monomorphic fixation probabilities 
by p AA , and p BB , respectively. Thus, the second condition, Eq. 
(14), should be interpreted as 

Pa>Pbb (16a) 



Pb>Paa> (16b) 

i.e. that the fixation probability of a single A (or B) mutant in a B 
(A) population exceeds that of one B (A) individual turning into the 
common ancestor of the entire population. 

If mutations occur during an updating event, then in 
heterogeneous populations mutants occur more frequendy in 
some vertices than in others. For our imitation process, high 
degree vertices serve more often as models than low degree 
vertices and hence the mutation is likely to occur in neighbours of 
high degree vertices. Note that this is different from placing a 
mutant on a vertex chosen uniformly at random from all vertices 
[47]. A randomly placed neutral mutant fixates, on average, with a 
probability corresponding to the inverse of the population size. 
This is not necessarily the case if neutral mutants arise in 
reproductive events or errors in imitating or adopting other 
strategies. In fact, the distinction between pQ,p AA and p BB is only 
required on heterogenous graphs with accumulated payoffs and 
non-random locations of mutants. In all other situations the 
(average) monomorphic fixation probabilities are the same and 
equal to p 0 = 1 /A, where TV is the population size. 

In summary, due to the fitness differences in a monomorphic A 
population with accumulated payoffs the turnover is accelerated 
and more strategy updates take place and hence more errors occur 
than in the corresponding monomorphic B population. This 
means that, on average, mutant Bs more frequendy attempt to 
invade an A population than vice versa. Overall, this leads to new 
conditions for evolutionary success in heterogeneous populations, 
summarized as follows. Type A (i) has an evolutionary advantage or 
is favoured if p A p A > P-bPb where p A is the probability a mutant A 
arises in an all-Ti population (and vice-versa), and, is beneficial if 
Pa > Pbb> where p BB is the probability a single B individual goes 
on to become the common ancestor in an all-i? population. 
Analogous conditions hold for a mutant B type. We apply these 
novel conditions to an example found in the literature [47], the 
star graph. 

The star graph. The star graph represents the simplest, 
highly heterogenous structure. A star graph of size N + 1 consist of 
a central vertex, the hub, which is connected to all N leaf vertices. 
On the star graph the range of degrees is maximal - the hub has 
degree N and all leaves have degree one. 

In order to illustrate the differences arising from accumulating 
and averaging payoffs, consider a situation where each individual 
initiated, on average, one interaction. Thus, the hub has A + 1 
interactions while the leaves have only 1 + 1 /A. Assume that i 
vertices are of type A and A — i of type B. The payoff to a hub of 
type A is then (i + (N — l)S)(l + 1/JV) for accumulated payoffs 
and (i + {N — i)S)/N if payoffs are averaged. In contrast, the 
payoff of an A leaf is l + l/N (accumulated) and 1 (averaged). 
From each A leaf the hub gains 1 + 1 / A for accumulated payoffs, 
which is the same as the gain for the A leaf. However, for averaged 
payoffs, the hub only gains 1 /N from each A leaf but each A leaf 
still gains 1 from the interaction with the hub. Thus, A-A- 
interactions are more profitable for vertices with a low degree and 
the payoff gets discounted for vertices with larger degrees. 
Although potential losses against B leaves also get discounted: T 



for B leaves versus S/N for an A hub for averaged payoffs as 
opposed to 7X1 + 1 /A) for B leaves versus 5(1 + 1 /A) for an A 
hub for accumulated payoffs. For A types it is less attractive to 
interact with B types whenever S < 1 and hence applies to all 
generalized social dilemmas [12]. 

Similarly, the payoffs to a type B hub are i T(l + 1 / A) 
(accumulated) and iT/N (averaged) versus 0 for B leaves 
(accumulated and averaged) or S(l + l/N) (accumulated) and 
S (averaged) for A leaves. In 5-i?-interactions both players get 
zero, regardless of the aggregation of payoffs, which is a 
consequence of our particular scaling of the payoff matrix in 
Table 1. Hence there is no discrimination between vertices of 
different degrees. An illustration of the differences arising from 
payoff accounting for the simpler and more intuitive case of the 
prisoner's dilemma in terms of costs and benefits (see Table 2), is 
given in Figure 4. 

In particular, on star graphs or, more generally, on scale-free 
networks, averaged payoffs result in higher and hence less 
favourable cost-to-benefit ratios for most individuals in the 
population, those with the lower degree vertices. Naturally these 
differences are also reflected in the evolutionary dynamics. We 
demonstrate this through the fixation probabilities of a single A (B) 
type in a population of B (A) types. 

Let us first consider the fixation probability of a single A 
type, p A . Because of the heterogenous population structure, p A 
depends on the location of the initial A - for a star graph, 
whether the A originated in the hub or one of the leaves. We 
denote the two fixation probabilities by p A \ H and p A n, 
respectively. With probability N/(N+Y) one of the leaves is 
chosen to update its strategy and the hub with probability 
l/(N+ 1). For averaged payoffs the fitnesses of everyone is the 
same in a monomorphic B population and hence the hub is 
equally likely to adopt the strategy of a leaf, and make a 
mistake with probability p«l, as are leaves that are adopting 
the hubs strategy. Hence the average fixation probability is 
given by 

N I 

p ^ = n+i Pa ^ l+ n+i Pa ^ (17) 

In contrast, for accumulated payoffs even in a homogenous 
population the hub does not necessarily have the same payoffs 
as the leaves because of the larger number of interactions. 
However, for our payoff matrix in Table 1, this does not 
matter for homogenous B populations as all £-5-interactions 
yield a payoff of zero. Consequently, Eq. (17) equally holds for 
averaged and accumulated payoffs and, incidentally, this is also 
the average fixation probability for a randomly placed A 
mutant. 

Similarly, we are interested in the average fixation probability, 
p B , of a single B type in an otherwise homogenous A population. 
Again we first need to determine with what probability the B 
mutant arises in a leaf or in the hub. Interestingly, and in contrast 
to p A , this now depends on the accounting of payoffs. If payoffs are 
averaged then all individuals have the same payoff and, in analogy 
to Eq. (17), we obtain 

= aTT Pbil+ aTT Pm ' (18) 

However, for accumulated payoffs, the hub achieves a payoff of 
A + 1 as compared to an average payoff of merely 1 + 1 / A for the 
leaves. In order to determine the average fixation probability of a 
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o • 




Figure 4. A star graph has the hub in the centre surrounded by N leaf vertices. Using the matrix in Table 2, an A type individual (blue) on 
the hub provides a benefit b to each leaf, regardless of whether the payoffs are a accumulated or b averaged. For each interaction, the costs to the 
hub amount to c in the accumulated case whereas only c/N in the averaged case. Conversely, the costs to a type A leaf are always c and it provides a 
benefit b to the hub if payoffs are accumulated whereas only b/N when averaged. Hence for averaged payoffs an A type hub provides a benefit to 
each leaf at a fraction of the costs while A type leaves provide a fraction of the benefits to the hub. This means that the leaves and the hub are 
playing different games. More specifically, the cost-to-benefit ratio of A leaves is Nc/b while it is c/(Nb) for an A hub. For most of the population (the 
leaves), this ratio is much larger than for accumulated payoffs where the cost-to-benefit ratio is c/b. As a consequence cooperation is much more 
challenging if payoffs are averaged rather than accumulated. 
doi:10.1371/journal.pcbi.1003567.g004 



single B type, pf CCU , we first consider the case where the mutant 
arises on a leaf. With probability N/(N+ 1) a leaf is selected to 
update its strategy and adopts the hub's strategy with probability 
l/(l+exp(-<5(7V-l/JV))) (c.f. Eq. (10a)). If the leaf adopts the 
strategy it makes an error with a small probability and instead of 
copying the A strategy, the leaf becomes of type B. Similarly, the 
hub reassesses its strategy with probability l/(N + 1) and switches 
to the leafs strategy with probability 1/(1 +exp(<5(7V— \/N))), 
which may then give rise to an A type in the hub with a small 
probability. Based on these probabilities we can now determine 
the proportion of mutants that occur in the leaves and the hub, 
respectively. For the leaves we get 



A' 



1 



s(n-1 

l+e V N 



N 



1 



1 



1 



l + e V N 



N+l 

l+e V N 



N 



N + e 



61 N- 



N 



and similarly for the hub 
1 

N+l 



l+e 



S\ N- 



N 



N 
N+l 



1 



51 N- 



1 



1 



l+e 
1 



N 



l+e V N 



1+iVe 



SlN -N 



Thus, the average fixation probability of a single B mutant is 



pit- 



N 



N + e 



-5\ N — — 



Pb\l- 



l+Ne 



dlN ~N 



Pb\h- 



(19) 



In the weak selection limit, <5«1 (or, more precisely, <5iV« 1), 
Eq. (19) takes on the same form as for averaged payoffs, Eq. (18). 
Conversely, for large populations, 6N»l, mutants almost surely 
arise in leaves and hence p| ccu ~Pb\l- Note that this is a good 
approximation as for N = 100 and 5 = 0.1 the probability that the 
mutant arises in the hub is already less than 10 . 

In order to determine the evolutionary advantage of A and B 
types we still need to determine the rates Pa>Pb at which A and B 
mutants arise in monomorphic B and A populations, respectively. 
If payoffs are averaged all individuals in the population have the 
same fitness and hence with probability 1/2 the focal individual 
imitates its neighbour (c.f. Eq. (10a)) and with a small probability j.1 
an error (or mutation) occurs. This holds for monomorphic 
populations of either type and hence f.i A = fi B . For accumulated 
payoffs the same argument holds for monomorphic B populations 
where all individuals have zero payoff. Consequently, A mutants 
arise at a rate j.i A = l/2^i. In contrast, in a monomorphic A 
population the hub has a much higher fitness and leaves will 
almost surely imitate the hub (whereas the hub almost surely will 
not imitate a leaf): 



Pb~- 



\ 



N 



N+l 



\ 



l+e 



-6\ N — — 



+ 



N+l 



/x.(20) 



For large N every update essentially results in one of the leaves 
imitating the hub, so that p.gXf.L. 
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Equations (17) through (19) yield the conditions under which 
type A or B has an evolutionary advantage. For star graphs, the 
fixation probabilities, p A and p B , can be derived based on the 
transition probabilities to increase or decrease the number of 
mutants by one and hence the results can be easily applied to any 
update rule [47] . For the imitation dynamics A types are favoured 
under weak selection if and only if 



averaged 



_avs 
Pa 



-ave 
>Pb 



N-l 
IN 



>T-S 



accumulated : 

-accu^ -accu 2iV(jV+l) 



>T~S 



(21a) 



(21b) 



and in the limit of infinite populations, TV— >co, the conditions 
reduce to 



averaged: f A g >p B yg <>->T -S (21c) 



accumulated ; 



^accu -accu 
PaPa >PbPb 



ol>T-S 



(21d) 



fixation probabilities in (22a) through (23) corroborate the 
approximation results of [23] and the analytical results of [21]. 

Let us now turn to the monomorphic A population and 
determine p AA . If <5 = 0 then everything is the same as in the 
monomorphic B population above and p AA = l/(N+ 1). Howev- 
er, for any non-zero selection, <5>0, the situation becomes more 
interesting. If payoffs are averaged, all individuals have the same 
(non-zero) payoffs and a mutant is equally likely to appear in the 
hub as any particular leaf (c.f. Eq. (18)) and hence p AA = l/(N + 1) 
still holds. However, if payoffs are accumulated the hub has a 
higher fitness. The fixation probabilities that an A on the hub or 
one of the leaves becomes the common ancestor are Paa\h ar >d 
Paa\l ( see Materials and Methods) and, on average we obtain 



-accu 
Paa - 



1 



1 {N-l 



N+l 2\N+l 



s+o(s 2 



(24) 



Now we are able to derive the conditions under which an A and/ 
or B mutant is beneficial, c.f. Eq. (16): 



_avg -avg 
Pa Pbb 

(4N 2 - 3iV - 1) + (147V 2 - 3N+ 1)5 >( ION 2 + 3N - l)T 



(25a) 



A detailed derivation of the different fixation probabilities is 
provided in the Materials and Methods Section. 

In order to determine whether a mutant is favoured or not (see 
Eq. (16)), we first need to determine the fixation probabilities p AA 
and p BB . Naturally, those fixation probabilities again depend on 
whether the ancestor is located in the hub or one of the leaves. 
Let us first consider a monomorphic B population. The fixation 
probability of a B located in the hub, Pbb\h> or m one particular 
leaf, Pbb\l> can be derived from the fixation probabilities p B \ H 
and p B \ L by setting f, = l (see Materials and Methods), which 
yields 



Pbb\h z 



1 



Pbb\l~^- 



(22a) 



(22b) 



becomes the common ancestor 
any leaf individual updates its 



Intuitively, the hub individual 
with probability 1/2 because 
strategy to the hub's with a probability of 1 /2 and the hub keeps 
its strategy also with probability of 1 /2 but both probabilities are 
independent of the size of the population. Conversely, a leaf 
individual must first be imitated by the hub, which is l/N times 
less likely than the reverse. On average we then obtain (insert into 
Eq. (17)): 



_avg -avg 
Pb B >Paa B<> 

(W 2 -9N+l) + (lON 2 + 3N-l)S<(UN 2 -?,N+l)T 
for averaged payoffs and, for accumulated payoffs, 



-accu^ -accu 
Pa >Pbb 

(4N 2 — 3N — l) + (5N 2 + 9N — 2)S>(N 2 + 15N — 4)T 



^accu^ ^accu^ 
Pb >Paa <s> 

(8 A" 2 - 9 N + 1 ) + (N 2 + 1 5N - 4)5 < (57V 2 + 9 TV - 2) T. 



(25b) 



(25c) 



(25d) 



The parameter region which delimits the region of evolutionary 
success of A and B types is illustrated in Figure 5. 

We can analyze Eqs. (21a)-(21d) and (25a)-(25d) in terms of the 
additive prisoner's dilemma game by substituting S = —c/(b — c) 
and T = bj(b — c). For simplicity, we restrict attention to the case 
iV->co and since in the additive prisoner's dilemma game a 
strategy is favoured if and only if it beneficial we need only 
consider Eqs. (21a)-(21d). We have 



, -avg _avg o _ 
averaged : p A >p B o - < — 3 



(26a) 



Pbb z 



1 



N+l' 



(23) 



Note that in a monomorphic B population the payoffs are zero 
regardless of the selection strength, 5, location (hub and leaves) or 
the payoff accounting. Again, this is a consequence of our 
particular choice of payoff matrix (Table 1), and thus, Eq. (23) 
holds for both averaged as well as accumulated payoffs and is, in 
fact, the same as the neutral fixation probability p 0 . Note that the 



accumulated : fi A p A ccu > fi B p B C 



>3 



(26b) 



If we suppose b,c>0, then Eq. (26a) is never satisfied. That is, 
averaging rather than accumulating the payoffs altogether 
removes the ability of the star graph to support cooperation. 

Note that for additive, or equal-gains-from-switching, games 
(games that satisfy S+ T= 1) and for weak selection the condition 
Pa>Pbb implies both p B <p AA and p BB = p AA = \/{N +1), 
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accumulated averaged 




S S 

Figure 5. Criteria for evolutionary success on the star graph for accumulated (left column) and averaged (right column) payoffs for 
weak selection, S«l. The range for which A is advantageous (top row, c.f. Eq. (15)) depends on the population size, N, and is shown in the limit 
N-»ao (solid line) and for TV = 5 (dashed line). Below the respective lines A is favoured. Similarly, the range for which A and B mutants are beneficial 
(c.f. Eq. (16) also depends on N. B mutants are beneficial above the red lines, while A mutants are beneficial below the blue lines (solid for N-*(X>; 
dashed for N = 5). Additive games (or equal-gains-from-switching) satisfy S+ T— 1 (dotted line). 
doi:10.1371/journal.pcbi.1003567.g005 



regardless of the accounting of payoffs. This extends results 
obtained for homogenous populations [8,10]. 

Stochastic Interactions and Updates 

As we have seen, when payoffs are averaged, members of a 
heterogeneous population are possibly playing different games, 
while if they are accumulated, all individuals play the same game. 
Therefore, only accumulating payoffs allows for meaningful 
comparisons of different heterogeneous population structures. A 



common simplifying assumption is that each individual interacts 
once with all its neighbours, see Figure 3. For heterogeneous 
populations this assumption means that those individuals residing 
on higher-degree vertices are interacting with their neighbours at a 
higher rate than those on lower-degree vertices. This leads to a 
separation of time scales, where interactions occur on a much 
faster time scale than strategy updates. 

Realistically, all social interactions require a finite amount of 
time and hence the number of interactions per unit time is limited. 
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Figure 6. Average fraction of strategy A for different ratios between interactions and strategy updates in homogenous (top row) 
and heterogeneous (middle row) populations and the difference between them (bottom row) as a function of the game parameters 
S and T (c.f. Figure 3). Interactions occur with probability to and strategy updates with 1— co. For example, for co = 0.9 each individual has, on 
average, initiated 9 interactions between strategy updates but only an average of 1/9 interactions for e»=0.1. For small co effects of heterogenous 
population structures have little chance to manifest themselves and the results are closer to those for averaged payoffs (c.f. Figure 3). In contrast, for 
large co heterogeneity plays an important role: for scale-free networks it is guided by the structural heterogeneity whereas in homogenous 
populations another form of heterogeneity spontaneously emerges in the number of interactions. Even on lattices, stochastic differences in the 
number of interactions get amplified by the dynamics because an increased number of interactions reduces the chances that an individual updates 
its strategy (c.f. Figure 7). As a consequence the results for lattices and scale-free networks become increasingly similar but scale-free networks keep 
promoting A types to a greater extend. Parameters and averaging technique are as in the caption to Figure 3. 
doi:10.1371/journal.pcbi.1003567.g006 



This constraint already affects the evolutionary process in 
unstructured populations [48] but becomes particularly important 
in heterogenous networks where, for example, in scale-free 
networks some vertices entertain neighbourhood sizes that are 
orders of magnitude larger than that of other vertices. For those 
hubs it may not be possible to engage in interactions with all 
neighbours between subsequent updates of their strategy or the 
strategies of one of their neighbours. In order to investigate this we 
need to abandon the separation of the timescales for interactions 
and strategy updates. 

A unified time scale on which interactions and strategy updates 
occur can be introduced as a stochastic process where a randomly 
chosen individual i initiates an interaction with probability a> with 
a random neighbour j and reassesses its strategy with probability 
1 — co by comparing its payoff to that of a random neighbour 
according to Eq. (9). Interactions alter the payoffs %t,ltj of both 
individuals (and hence their fitnesses, //,//, see Eq. (la)) according 
to the game matrix in Table 1 . If individual i adopts the strategy of 
its neighbour, then its payoff (and interaction count) is reset to 
zero, 71/ = 0, regardless of whether the imitation had resulted in an 
actual change of strategy. Simulation results for various co are 
shown in Figure 6. 

For small co«l few interactions occur between strategy 
updates and in the limit co-»0 neutral evolution is recovered 
because no interactions occur. Conversely, in the limit co— >1 



many interactions occur between strategy updates, which 
allows individuals to garner large payoffs as well as build up 
large payoff differences. The average number of interactions 
initiated by any individual between subsequent reassessments 
of the strategy is co/(l — co), the relative ratio of the time scales 
of game interactions versus strategy updates. However, the 
distribution of the number of interactions is biased: individuals 
with a large number of interactions tend to score high payoffs 
and hence are less likely to imitate a neighbours' strategy, 
which in turn results in a further increase of interactions. On 
heterogenous graphs and scale-free networks, in particular, this 
bias is built in by the underlying structure because highly 
connected hubs engage, on average, in a much larger number 
of interactions than vertices with few neighbours. Moreover, 
hubs are more likely to serve as models when neighbours are 
reassessing their strategy - simply because hubs have many 
neighbours. Thus, hubs are not only more resilient to change 
but also have a stronger influence on their neighbourhood. 
When co/(l— co) this ratio begins to get large, interactions 
dominate strategy updates and the resulting game dynamics on 
heterogeneous and homogeneous graphs becomes indistin- 
guishable. 

Interestingly, a similar bias in interaction numbers sponta- 
neously emerges on homogenous graphs, lattices in particular. 
Since all vertices have the same number of neighbours, no 
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Figure 7. Distributions of the number of interactions on lattices (black ) and scale-free networks (blue A) with a few interactions 
between updates (co = 0.02 or, on average, «0.02 interactions) and b many interactions between updates (co = 0.98 or, on average, 49 
interactions). For small co the heterogeneity of scale-free networks results in a pronounced tail at higher numbers of interactions compared to the 
approximately exponential distribution for lattices. This tail is responsible for the reduction of cooperation in scale-free networks observed in Figure 6: 
as interactions dominate, some vertices almost never update their strategies. This "static network" emerges in both lattices and scale-free graphs and 
prevents the complete proliferation of the rare strategy. Nevertheless, most of the individuals in the population experience essentially the same 
number of interactions. The distributions look different for large co but the main difference remains that scale-free networks produce a more 
pronounced tail. More importantly, however, for most of the population the distributions are actually very similar and hence the heterogeneities very 
similar. On lattices, the skewed distribution is caused by stochastic variations and the positive feedback between the number of interactions and the 
resilience to changing strategy. 
doi:10.1371/journal.pcbi.1003567.g007 



vertices are predisposed to achieve more interactions than 
others but some inequalities in interaction numbers occur 
simply based on stochastic fluctuations. As above, those 
vertices that happen to engage in more interactions tend to 
have higher payoffs and hence are less likely to imitate their 
neighbours and keep aggregating payoffs. This positive 
feedback between interaction count and resilience to change 
spontaneously introduces another form of heterogeneity, which 
becomes increasingly pronounced for larger co. In fact, for 



large oj it rivals the structurally imposed heterogeneity of scale- 
free networks, see Figure 7. 

Regardless of the structure, the positive feedback between 
payoff aggregation and the diminishing chances to change strategy 
(and hence reset payoffs) means that a small set of nodes forms an 
almost static backdrop of the dynamics and hence has a 
considerable effect on the evolutionary process. This set is a 
random selection on homogenous structures and consists of the 
hubs on heterogenous structures. As a consequence, the initial 
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Figure 8. Impact of the time scale relation co on the equilibrium fractions of cooperators in the Si i-plane for additive prisoner's 
dilemma games (T= 1 — S): a lattices and b scale-free networks. The limit co->0 recovers the neutral process (no interactions) whereas for 
co-> 1 individuals hardly update their strategies. Thus, in both of the two limiting cases the fraction of cooperators remains at the initial value of 0.5. 
For both types of population structures there exists an intermediate co that leads to an optimal level of cooperation. On lattices the support for 
cooperation is strongest if interactions and strategy updates occur at equal rates, co = 0.5, but on scale-free networks more frequent updates than 
interactions are even more beneficial, cos= 0.25. Parameters and averaging technique are as in the caption to Figure 3. 
doi:10.1371/journal.pcbi.1003567.g008 
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configuration of the population has long lasting effects on the 
abundance of strategies. 

A more detailed view on the effects of co on the evolutionary 
process is provided by restricting attention to the prisoner's 
dilemma and additive payoffs, c.f. Table 2. This can be 
accomplished by setting T=l — S with S<0. The equilibrium 
levels of cooperation in the So-plane are shown in Figure 8 for 
lattices and scale-free networks. 

Altering the relative rates of interactions versus strategy updates 
has interesting effects on the evolutionary outcome. For lower rates 
of interaction (co« 1), scale-free networks outperform lattices in 
their ability to promote cooperation. As interaction rates increase 
and strategy updates become more rare (cowl), scale-free 
networks and lattices become virtually indistinguishable in their 
ability to support cooperation. For both lattices and scale-free 
networks an optimal ratio between strategy updates and interac- 
tions exist: for lattices this is roughly CO =1/2, suggesting that 
lattices support the greatest amount of cooperators when 
interactions occur at the same rate as strategy updates, whereas 
for scale-free networks the optimum lies around co«0.25, which 
suggests that scale-free networks provide the strongest support for 
cooperation if there are roughly three updates per interaction. 

Discussion 

Evolutionary dynamics in heterogenous populations, scale-free 
networks in particular, have attracted considerable attention over 
recent years. Somewhat surprisingly, the underlying microscopic 
processes and their implications for the macroscopic dynamics and 
the corresponding biological interpretations have received little 
attention. 

Here we have shown that established criteria to measure success 
in evolutionary processes make different kinds of implicit 
assumptions that do not hold in general for heterogenous 
structures. Instead, for such structures it becomes imperative to 
reconsider, revise and generalize these criteria, which was done in 
the Criteria for Evolutionary Success section. If errors arise in 
imitating the strategic type of other individuals, or mutations occur 
during reproduction, then mutations are more likely to arise in 
some locations than in others. For example, on the star graph 
mutants likely occur in the leaf nodes for birth-death updating and 
imitation processes but in the hub for death-birth processes. 
Moreover, in heterogenous populations the fixation probabilities 
generally depend on the initial location of the mutant and hence 
even the fixation probability of a neutral mutant may no longer 
simply be the reciprocal of the population size but rather 
intricately depend on the population structure. 

Another crucial determinant of the evolutionary dynamics in 
heterogenous populations is the aggregation of payoffs from 
interactions between individuals. Individuals on vertices with a 
higher (lower) degree expect to have more (fewer) interactions than 
on average. Even though the choice between averaging or 
accumulating payoffs may seem innocuous, it has far reaching 
consequences. Previous authors [29] have found that averaging 
payoffs in a prisoner's dilemma game on a scale-free network 
eliminates such a network's ability to promote cooperation as 
observed in earlier studies [24,25,27,49]. We have extended this 
result to general 2x2 games and provide a detailed rationale for 
this phenomenon which is summarized as follows. If payoffs are 
accumulated, some individuals are capable of accruing more 
payoffs than others strictly by virtue of them having more potential 
partners. Averaging payoffs removes the ability of hubs to accrue 
greater payoffs, but simultaneously makes it difficult to compare 
results for different population structures (e.g. lattices versus 



scale-free networks) even if their average degrees are the same 
because the type of game played depends on the location in the 
graph. Hence, accumulating payoffs seems a more natural choice 
to compare evolutionary outcomes based on different population 
structures because it ensures that everyone engages in the same 
game. However, if we assume all interactions are realised then 
those individuals with more neighbours interact at a much greater 
rate than those with less. 

In order to investigate the disparity in the number of 
interactions on the success of strategies on heterogenous graphs 
we introduced the time-scale parameter co, which determines the 
probability that an interaction or a strategy update occurs. When 
increasing the rate of strategy updates (small co), heterogeneous 
graphs are able to support higher levels of cooperation than 
lattices. Conversely, increasing the rate of interactions (large co) 
results in small differences between lattices and scale-free 
networks; both support roughly the same levels of cooperation. 
For imitation processes, individuals with high payoffs are unlikely 
to change their strategies and hence are likely to keep accumu- 
lating more payoffs. On scale-free networks, hubs are predestined 
to become such high performing individuals but on lattices they 
spontaneously emerge, triggered by stochastic fluctuation in the 
interaction count and driven by the positive feedback between 
increasing payoffs and increasing resilience to changing strategies 
(and hence to resetting payoffs). 

For intermediate co an optimum increase in the level of 
cooperation is found: lattices support cooperation most efficiently if 
a balance is struck between interactions and strategy updates 
(wx0.5), whereas scale-free networks work most efficiently if slighdy 
more updates occur (co « 0.25). For lattices a related observation was 
reported for noise in the updating process [50] . If the noise is large, 
updating is random but if it is small the game payoffs become 
essential. Interestingly, cooperation is most abundant for intermediate 
levels of noise - which is similar to having some but not too many 
interactions between strategy updates. 

Previous work has found that heterogeneous graphs support 
coordination of strategies, where all individuals are inclined to adopt 
the same strategy, while homogeneous graphs support co-existence 
[51,52]. The time scale parameter co introduced in the Stochastic 
Interactions and Updates section seems to aid in promoting 
coexistence in both types of graphs, based on the large green 
region in Figures 3, 6, and 8. Exactly how the time scale parameter 
co promotes coexistence is a topic worthy of further investigation. 

Naturally there is no correct way of modelling the updating of the 
population or the aggregation of payoffs but, as so often, the devil is 
in the detail and implicit assumptions originating in traditional, 
homogenous models may be misleading or have unexpected 
consequences in more general, heterogenous populations. 

Materials and Methods 

In [47], the authors calculate expressions for the probability that 
a single mutant fixes on a star graph. These expressions are in 
terms of state transition probabilities. Denote by PfJ the 
transition probability from a state with i A individuals on the 
leaves and an X individual on the hub to a state with j A 
individuals on the leaves and a Y on the hub. With this notation, 
the fixation probability of a single A on a leaf vertex is 



PAlL pM+ptf*A(W 
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and for a single A on the hub, 

P?! 4 1 BB i e Z(ilN)T 

pA \ H= p BA + pBBA{\,N)' (28) Pi ^ = N+le^/N)T + e ss- ( 31d ) 

where, in both cases, These are incorporated into the Eqs. (27) and (28) to yW £* 

fixation probabilities p A \ L and Pa\h- The frxation probabilities > 

, . and p B , H are obtained in a similar way. The averages P^b 11 and 
JV-1 pAB , poB lpAA + pAB\ 

A(l N) = \ +^ hi pj K ' K ' V /CA+1 ^2 / (29) Pa b are tnen calculated using Eqs. (17), (18), and (19). Finally, a 

j = \ Pjj^ ' + Pjj+i k=\ P^k+i i^P^k-\ ^^kk^) first-order approximation in S is found for the above. For example, 



1 jv i d r i w i „ /s2 , 

JV+T "4» + jV+T^ |£ J,5 = 0 + ^[jV+T ^ + A?TT Hi = o + ( ' 



For the imitation process defined by Eq. 9 and accumulated 

_. . F y H 1 ( ( \ \d ( N \d 

payoffs we have = ^— + hrrT hrP^ L + vTT 



^TT T VV^Ty^' M " J b = 0 T V^T^' M|£ |«5 = o/ + 0< ' 5 ' (32a) 



^ _ N -i e S(i + (N-i)S)(l + l/N) + ( 4**-w-i)) + 0(4 >) 

P i,i+ 1 " N+l e S(i + (N-i)S)(l + l/N) +e ST(l + l/N) (3 ° a) 



The other fixation probabilities are found in a similar way: 



p AB = 

i,i 

1 N—i e S m + l/N) 

N+\ N e S(i + (N-i)S)(l + l/N) +e 3T(l + l/N) 



(30b) 
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(32b) 



p BA 



1 i 
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e biT(\ + \/N) 
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and for averaged payoffs, 
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(30d) 



(31a) 
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(31c) 
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((10N 2 + 3N-1)S-(14N 2 -3N+1)T 
+ (SW 2 -9jV+l))+0( ( 5 2 ) 
Assuming <5« 1, 

and employing the appropriate condition for evolutionary 
advantage, yields Eqs. (25a-25d) in the main text. 
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